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SQUARES.  THE  METHOD  OF  EVALUATING  THE  CONSTANTS 
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by 

1/  ?  / 
Robert  E.  Barieau-  and  B.  J.  Daltons 


ABSTRACT 

This  report  is  concerned  with  the  necessary  mathematical  equations 
in  order  to  accomplish  the  following  objectives:  1.  to  evaluate  the 
parameters  such  that  the  sum  of  the  weighted  squares  of  the  residuals 
of  an  experimental  observable  be  a  true  minimum,  regardless  of  the 
functional  relationship  between  the  variables  and  these  parameters; 

2.  to  evaluate  all  variances  and  covariances  of  the  parameters  eval¬ 
uated  by  means  of  the  law  for  the  propagation  of  errors;  and  3.  to 
insure  that  a  true  minimum  will  always  be  obtained.  The  equations 
presented  in  this  report  were  developed  for  a  three-parameter  problem. 

INTRODUCTION 

The  Helium  Research  Center,  Bureau  of  Mines,  has  as  one  of  its 
long-range  objectives  the  development  of  an  equation  of  state  for 
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helium  that  will  reproduce  the  data  to  within  the  accuracy  with  which 
the  data  are  known. 

The  Helium  Research  Center  also  has  an  experimental  program  j-or 

3  / 

obtaining  PVT  data  on  gases  and  mixtures  by  the  Burnett  (2)~  method. 


3/  Underlined  numbers  in  parentheses  refer  to  items  in  the  list  of 
references  at  the  end  of  this  report. 


In  the  Burnett  method,  ono  of  the  constants  that  must  be  evaluated  is 
the  volume-ratio  of  two  containers.  By  the  theory  of  the  method,  this 
constant  is  inherently  non-linear.  It  was  therefore  decided  to  develop 
a  capability  for  handling  non-linear  regression  problems.  This  report 
gives  the  principles  of  the  method  the  Helium  Research  Center  uses  in 
such  problems. 

Our  method  differs  in  several  important  respects  from  methods  cur¬ 
rently  in  use.  In  solving  non-linear  problems,  the  problem  must  be 
linearized  and  an  iteration  technique  used  to  obtain  the  solution.  All 
texts  on  non-linear  regression,  of  which  we  are  aware,  linearize  the 
problem  before  the  normal  equations  are  formed.  This  method  is  known 
as  the  Gauss-Newton  method.  In  our  method,  the  exact  normal  equations 
are  formed,  and  the  problem  is  linearized  by  expanding  the  exact  normal 
equations  in  a  Taylor's  series  expansion  retaining  the  first  two  terms. 
This  method  is  known  as  the  Newton-Raphson  method  (_7)  .  The  only  work 
that  we  have  been  able  to  locate  that  uses  this  method  in  non-linear 
least  squares  problems  is  that  of  Strand,  Kohl,  and  Bonham  (_8)  . 


These  two  different  methods,  as  will  be  shown  later,  lead  to  the 
same  least  squares  solution  provided  the  iteration  procedure  converges 
to  an  answer.  We  have  found  that  if  one  starts  within  the  region  of 
convergence,  the  Newton-Raphson  method  converges  more  rapidly  than  the 
Gauss-Newton  method.  This  is  one  advantage  of  the  method  we  have 
chosen . 

If  on  applying  the  Gauss-Newton  or  the  Newton-Raphson  method,  the 
problem  is  diverging  after  the  first  iteration,  a  method  must  be  found 
that  will  lead  to  convergence.  One  of  the  methods  that  may  be  tried 
at  this  stage  is  the  negative  gradient  or  method  of  steepest  descent. 

If  the  step  in  the  direction  of  the  negative  gradient  is  small  enough, 
this  method  must  lead  to  a  smaller  sum  of  the  squares  of  the  deviations. 
The  problem  with  this  method  in  the  past  has  been  deciding  on  the  size 
of  the  step  to  be  taken.  If  the  Newton-Raphson  method  has  been  used 
in  developing  the  normal  equations  for  the  first  iteration,  then  the 
size  of  the  step  to  be  taken  in  the  direction  of  the  negative  gradient 
can  be  evaluated  very  simply  from  the  coefficients  appearing  in  the 
normal  equations.  This  is  the  second  advantage  of  the  method  we  have 
chosen . 

The  third  advantage  involves  the  calculation  of  the  variances  and 
covariances  of  the  constants  evaluated.  As  far  as  we  are  aware,  all 
authors  and  all  programs  available  calculate  variances  and  covariances 
on  the  assumption  that  the  formulas  that  apply  to  linear  problems  will 
apply  to  non-linear  problems  once  the  non-linear  problems  have  been 
linearized.  We  reject  this  assumption,  preferring  to  calculate 
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variances  and  covariances  from  the  fundamental  definition  of  these 
quantities  and  the  law  for  the  propagation  of  errors.  (j,,  _5)  . 

Some  authors  (j6)  claim  that  the  least  squares  values  of  the 
constants  evaluated  in  a  non-linear  problem  are  biased  and  should 
be  corrected.  All  the  proofs  of  this,  that  we  have  seen,  assume 
the  deviations  are  distributed  with  zero  mean.  This,  of  course, 
is  never  true  in  a  non-linear  problem  unless  this  condition  is  im¬ 
posed  as  a  constraint.  Further,  the  principle  of  least  squares 
maximizes  the  probability  that  the  deviations  are  equal  to  the 
true  random  errors.  This  is  true  for  both  linear  and  non-linear 
problems.  This  being  true,  we  fail  to  see  how  any  solution  can  be 
better  than  the  least  squares  solution.  We  therefore  take  the 
least  squares  solution  as  being  non-biased  and  apply  no  correction. 

We  have  set  the  following  objectives  for  our  method. 

1.  To  evaluate  the  parameters  so  that  the  sum  of  the  weighted 
squares  of  the  residuals  of  an  experimental  observable  is  a  true 
minimum. 

2.  Objective  1  is  to  be  accomplished  even  though  the  func¬ 
tional  relationship  between  the  observables  and  parameters  is  such 
that  the  observable  involved  in  the  minimum  of  the  sum  of  the 
weighted  squares  of  the  residuals  cannot  be  explicitly  expressed 
as  a  function  of  other  observables  and  the  parameters. 

3.  All  variances  and  covariances  are  to  be  calculated,  with 

no  approximations,  by  means  of  the  law  for  the  propagation  of  errors. 

4.  The  method  is  to  be  such  that  a  true  minimum  will  always  be 


obtained . 
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This  report  is  concerned  with  the  necessary  mathematical  equations 
in  order  to  accomplish  the  above-named  objectives.  The  equations  are 
developed  for  a  three -parameter  problem.  The  extension  to  the  evalua¬ 
tion  of  more  parameters  should  be  obvious. 


EVALUATION  OF  THE  CONSTANTS 

Suppose  one  has  experimentally  determined  a  set  of  n  data  points 
Xf,  y  .  Let  the  functional  relationship  between  x  and  y  and  the  para¬ 
meters  A,  B,  and  C  be  given  by 

F(y,x,A,B,C)  =  0  (1) 

We  assume  that  there  are  no  random  errors  in  the  x.'s  and  that 

i 

random  errors  occur  in  the  observed  y.’s. 

J  l 

Now  suppose  we  have  evaluated  the  constants  A,  B,  and  C  by  some 
means  or  other.  Then,  because  of  random  errors  in  y  ,  equation  (1) 
will  not  be  exactly  satisfied  when  the  observed  y.  and  x.  values  are 

1.  X- 

substituted  in  equation  (1) .  We  will  let  F,  be  the  numerical  value  of 
F,  when  the  observed  y^  and  x^  values  are  substituted  in  equation  (1) . 
Thus  , 


Fi  =  F(yi(o)’xi’A>B’C) 


(2) 


Now  when  x^  is  substituted  in  equation  (1)  ,  we  may  solve  for  y_^  so 

that  equation  (1)  is  satisfied  exactly.  We  will  designate  this  y^ 

as  y,,  ,  x .  Thus , 

^i(calc) 

F(y .  .  .  v  ,x . ,A,B ,C) 

J i(calc)  l  5  ’ 

The  residual  of  y^  is  given  by 


0 


(3) 


. 
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i(o)  yi(calc) 


(4) 


Now  Y..  the  residual  of  y . ,  is  the  difference  between  the  observed 
1  J  i 

and  calculated  values.  This  is  not  the  true  random  error  in  our 

observed  because  we  do  not  know  the  true  value  of  y  .  However, 

we  can  maximize  the  probability  that  our  Y^'s  are  equal  to  the  true 

random  errors,  and  this  is  just  what  the  principle  of  least  squares 
* 

does.  The  principle  of  least  squares  says  that  we  maximize  the 
probability  that  the  Y^'s  represent  the  true  random  errors  by  mini¬ 
mizing  the  sum  of  the  weighted  squares  of  the  residuals. 

Thus,  the  function  to  be  minimized  is  given  by 


n 


=  i-. 


V2 


i  =  l 


i(o)  1 


(5) 


where  w 


is  the  weight  assigned  to  y^^  *  R  is  a  function  of 


yi(o) 

the  constants  to  be  evaluated:  A,  B,  and  C.  The  condition  that  R 


be  a  minimum  is  determined  by 


J, 

2 


n 


-dY. 


-  y«  Y.fci)  - 

B.C  &  yi(o)  A3A  ' 


0 


(6) 


2~(ft)  ‘  IWy  Yi(a r)  =  ° 

A,C  i=i  ^i(o) 


(7) 


and 


n 


.dY., 


2"(i).  „  ■  Iwy..^Yi(ar)  = 


i=l 


i(o) 


0 


(8) 
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In  the  application  of  equations  (6) ,  (7) ,  and  (8) ,  the  observed 

y,'s  and  x.'s  are  to  be  held  constant  in  the  derivatives: 
i  i 

’  •  Equations  (6)  ,  (7)  ,  and  (8)  are  the 

B,C  VSB/A,C  VdC/A,B 
exact  normal  equations. 

If  Y.  is  non-linear  in  the  undetermined  constants,  then  the 
l 

solutions  of  our  normal  equations  will  not  be  straightforward. 

It  will  be  necessary  to  solve  them  by  an  iterative  technique  in 

which  values  of  the  constants  are  assumed.  This  is  done  by  ex- 
/3Yi\  /dYA  /dYi\ 

panding  Y^,  5  \3B/’  anc*  VdC  /  3  Tay^or's  series  expan- 


.o 


sion  about  an  approximate  solution,  Y  ,  retaining  only  the  first 
two  terms.  Thus, 


Y. 

l 

■  Y°  +  ( 

^Y; 

^A 

. \  o  ,dY., 

-)  “  +  (as1. 

,°  /BY  < 

)  aB  +  (ac1) 

D 

AC 

(£) 

■  C‘T 

,a2v  .  o 

,B2Y., 

o 

2 

+  ( 

,  2  )  M  +  < 
A 

I  AB  +  ( 

^AdC/ 

(?) 

■  & 

,a2y  .  ° 

,B2Y iX 

o 

,b2y., 

+  ( 

^bba)  m  +  ( 

I  AB  +  ( 

sdBdC/ 

,dY.\ 

,dY.\0 

?Y,o 

,a2Y 

o 

2 

,d  Y.v 

fe1) 

■  (*■) 

+  ( 

.seal)  *  +  ( 

vdCdB/ 

1  AB  +  ( 

(9) 


j  AC  (10) 


AC  (12) 


where  the  quantities  AA,  AB ,  and  AC  are  defined  as 

AA  =  A  -  A 


o 


AB  =  B  -  B 


o 


AC  =  C  -  C 


o 


' 
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where  A,  B,  and  C  are  our  undetermined  constants,  and  Aq ,  B  ,  and 
are  approximate  values  for  these  quantities.  Then  to  first  order 
in  the  A's 


« 
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Substituting  equations  (13)  ,  (14)  ,  and  (15)  into  our 
equations  (6) ,  (7) ,  and  (8) ,  we  have 


ajAA  +  b;.AB  +  c^AC 
a2AA  +  b£AB  +  c^AC 
a^AA  +  b^AB  +  c^AC 


where 


normal 

(16) 

(17) 

(18) 

(19) 

(20) 

(21) 

(22) 

(23) 

(24) 


12 


n 


m. 


-BY.*  o 


w  4ar) 


i=l  yi(°) 


n 


v°/  i 

wy</ ,  YA^r 


BY  ,  o 


i=l  i(o) 


n 


m. 


BY.  o 


w  Y 

yj-(°) 


Aac  / 


(25) 


(26) 


(27) 


The  solutions  to  equations  (16) ,  (17) ,  and  (18)  are 


where 


D  AA 
o 


Dlmi  +  D2m2  +  D3m3 


D  AB  =  D.m.  +  Dm  +  D  m0 
o  4  1  5  2  6  3 

D  AC  =  D  m.  +  D  m  +  D  m„ 
o  7  1  8  2  9  3 


D1 

b2°3 

-  b3C2 

D2 

= 

Vi 

'  blC3 

D3 

= 

blc2 

-  b2c: 

D4 

= 

a3C2 

a2c3 

D5 

= 

aic3 

a3cl 

D6 

S 

a2cl 

alC2 

D7 

= 

a2b3 

a3b2 

00 

Q 

= 

a3bl 

'  alb3 

Dg 

- 

alb2 

-  a2b. 

(28) 

(29) 

(30) 

(31) 

(32) 

(33) 

(34) 

(35) 

(36) 

(37) 

(38) 


(39) 


, 
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D 


(40) 


o 


The  solutions  of  equations  (28)  ,  (29)  ,  and  (30)  give  the  corrections 
to  be  applied  to  the  assumed  values  of  our  undetermined  constants. 

In  the  Gauss -Newton  method  of  linearization,  the  second  term 
in  the  summations  of  the  a's,  b's,  and  c's  of  equations  (19) -(24)  is 
neglected;  this  does  not  lead  to  an  error  as  long  as  the  method  con¬ 
verges  because  the  exact  solutions  of  equations  (16),  (17),  and  (18) 
are:  AA  -  AB  =  AC  =  0. 

When  the  functional  relationship  between  the  observables  is  such 
that  yi(.calc^  cannot,  be  solved  for  explicitly,  it  will  be  necessary 


Let  us  expand  F  in  a  truncated  Taylor's  series  expansion  about  the 


F 


or 


F. 

i 


where  in  equations  (41)  and  (42) 


' 
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the  derivative  of  F  with  regard  to  y,  keeping  x,  A,  B,  and  C  constant 
is  to  be  evaluated  at  the  point 


y  =  yi(o) 


x  =  x 


The  solution  of  equation  (42)  gives  the  first  approximation  for 


./  \  •  We  designate  this  value  as  y.,  ,  .  . 

i(calc)  &  7i(calc)1 


then  substituted  into  equation  (3),  and  if  y. 


i(calc) 


This  value  is 

is  not  the  exact 


1 


answer,  equation  (3)  will  not  be  satisfied  exactly.  We  designate 

this  value  of  F  as  F.,  ,  .  .  Thus, 

i(calc)^ 


F 


i (calc) 


=  F(x  .  ,  y . 


i’7i(calc) 


,A,B,C) 


(43) 


Then  the  second  approximation,  y.,  ,  .  ,  of  y..  .  .  is  obtained 

i(calc) ^  i(caic) 

from  the  expression 


F. 


i(calc) 


yi(calc)  yi(calc)2 


1 


© 


x,A,B,C 


y=y. 


i(calc) 


x=x . 

l 


1 


(44) 


This  iteration  is  repeated  until  equation  (3)  is  satisfied  to  within 
any  amount  we  wish  to  specify. 


Once  we  have  y . ,  .  and  y  then  Y.  =y./N-y.,  .  N,  a 

i(calc)  i(o) ’  i  7 i (o)  7i(calc) 


nd 


Y.  can  be  calculated.  Then  if  w 

1  y 


i(o) 


is  known  or  has  been  assigned, 


R  may  be  calculated  by  means  of  equation  (4) . 


f 


■ 


' 
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If  the  y.^  's  all  have  the  same  precision  index,  they  will  ail 
have  the  same  weight  and  w 


yi(o) 

the  same  precision  index,  then 


=  1.  If  the  y .  ,  S's  do  not  all  have 

1(0) 


w 


y. 


(45) 


i(o) 


y. 


i(o) 


where  L  is  a  constant  and  S  is  the  variance  of  y . ,  N .  In  a 

yi(o)  l(o) 

particular  problem,  it  may  be  necessary  to  assume  that  w  =1 

yi(o) 

in  the  beginning.  However,  if  this  is  done,  the  residuals, 

-  y .  .  ,  v  ,  should  be  examined  to  see  if  there  is  any  statis- 

L  i(o)  •/i(calc)J 

ticai  evidence  for  the  residuals  squared  being  a  function  of  y.  Any 

2 

assumption  as  to  the  variance,  S  ,  being  a  function  of  y  can 

Yi(o) 

always  be  checked  by  examining  the  residuals.  In  any  event,  w 


y. 


i(°) 


is  not  a  function  of  the  constants  to  be  evaluated 


We  now  proceed  to  develop  the  equations  needed  to  calculate  the 
coefficients  of  our  normal  equations.  Differentiating  equation  (4) 
with  regard  to  A,  keeping  x^,  y^^  ,  B,  and  C  constant,  we  have 


\c)A  / 


x  .  ,y  .  ,B,C 
l  i(o) 


dy 


i(caic) 

SA 


) 


(46) 


x  ,B,C 


Differentiating  equation  (3)  with  regard  to  A,  keeping  x^,  B, 
and  C  constant,  we  have 


(sir) 


i(calc)  x^,A,B,C 


/dyi(calc)N\ 

\  SA  / 


+ 


x . ,B  ,C 

l 


x  .  ,y .  /  -  v,B,C 
i’-7i(caic) 


0  (47) 


. 


t 

■■ 


' 


1  ■ 

. 


‘:U 


y 
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or 


( 


dyi(calc)\ 

dA  ) 


x  ,B,C 


x . ,y . /  i  \ 
1  Ji(calc) 


SF 


) 


^yi(calc)  x . ,A,B ,C 


(48) 


and  if  we  substitute  equation  (48)  in  equation  (46)  ,  we  have 


/3Y-\ 

VdT)  B  c 

Xi’yi(o)’B’C 

Similarly,  it  can  be  shown  that 


and  that 


(?) 


Vyi(o)  >A’C 


© 


Vyi(o)’A’B 


x , ,y ,  /  i  \  jS 

l  i(calc) 


/  SF 

\ 

^yi(calc)‘ 

) 

x . ,A,B  ,C 

l 

(2E\ 

\SB  i 

x . ,y .  . 
r  'iC 

-  x  ,A,C 
calc) 

f  dF 

\ 

^yi(calc) 

Xf ’A’B >c 

© 

V’u 

1  x  ,a,b 

calc) 

(-22 - ) 


i(calc)  x^jAjBjC 


(49) 


(50) 


(51) 


Differentiating  equation  (49)  with  regard  to  A,  keeping  x^,  y^0^  > 
B,  and  C  constant,  we  have  . 


■*  l 
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c)2Y. 

i 


3A  xi,yi(o)’B,C 


(52) 


Now 


=  G 


Xi’yi(calc)’B>C 


lyi(calc) ,A,B,C) 


e(! 2 


X 


i,yi(calc) ,B’C  xi’B,C 


‘A  /’BF 

_dA  Vd 


.  x . 


dA 


+ 


-By 


.  ,y .  ,  ,  n,B,C  X..V.,  .  n,B.C 

i  •7i(calc)’  l  •'i(calc)  5 


^Yi(calc) 


i(calc)  x.,y.(calc);B,C  x.,A)B,C 


(53) 


and 


1  /dF\ 

l  KEl) 


b_  /A 
.5 A  \bAS 


Xi,yi(calc) ,B,C  Xi’B>c 


Xi ,yi(calc) ,B ,C  Xi ,yi(calc) ,B ,C 


+ 


LBy, 


b _  (b F 


i(calc)  x . ,y .  ,  ,  N ,B ,C  x . ,A,B  ,C 

l  J  i(calc)  *  i’  *  ’ 


bA 


Xl,B  ,C 


(54) 


1 


Also , 


(57^-) 


Syi(calc)V,A,B,C  V  i(calc) 


0 


=  ,a;,b ?c) 


'  yi(calc)  x. ,A,B,CJx.  ,B,C 


[' 


BF _ \ 


^yi(calc)  ^yi(caic)  x . ,A ,B ,C  x.,A,B,C 


^yi(calc) 


+ 


Cr^—)  I  dA 

ayi(calc)  x.,A,B,C  x.,y i(calc)’B’C 


and 


i(calc)  x  ,A,B,C  x^BjC 


d 


BF 


-) 


“  ^Italic)  *.,A,B,C 


fe“-r) 


^yi(calc)  ^yi(calc)  x^ ,A,B ,C  x^ ,A,B ,C 


^yi(calc)^ 


When  we  substitute  equations  (54)  and  (56)  into  equation.  (52) ,  we 


find 


' 


19 


B2y. 


L) 


dA  Xi’yi(o)’B’C 


(dy . 


BF  ^ 


.(5) 


dyj(calc)  x..,A,B,C  dA2  >yi(caic) >B >C 


( 


BF  \ 

^yi(caic)  x.,A,B,C 


SF . )  (--.SILL - ) 

dyi(calc)  x. ,A}B,CVSyi(calc)3  x.  ,B,CV  3A  '  x,  ,B,C 


/5yi(calc)^ 


(dy . 


BF  Y 


^yi(calc)  x^,A,B,C 


'BF\  (  B  F 

.BA/  „  „\BABy 

x . ,y .  ,  .  s  ,B  ,C 

1  i(calc) 


-) 


i(calc)  x^,B,C 


(sy. 


BF  Y 


^yi(calc)  x_^,A,B,C 


,( 


B2F 


Xi?yi(calc)  ,B?C  dyi(calc)  Xj?A’B’C 


x . ,B  ,C 

l 


( 


BF  \2 


^yi(calc)  x.,A,B,C 


But  from  equation  (48) 


^dyi(calc)^ 


x .  ,B  ,C 

l 


xi’yi(calc)’E’C 

(—22 - ■) 

^yi(calc)  x.,A,B,C 


(48) 
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] 


] 

] 

3 

3 


When  we  substitute  equation  (48)  in  equation  (57) ,  we  see  that 


A  Vyi(0)’B’C 


3 

3 
3 
3 
3 

3 
3 
3 


(M) 


'BA  '  x  .  ,y .  ,  .  \  ,B,C 

l  7i(calc) ’  J 


G 


B  F 


-) 


(—22 - ) 


x.,y.,  -  s ,B.C ’SASyi(caic)'  x . ,B,C 

l  i(calc) _ 2 _ l 


i(caic)  x^,A,B,C 


( — M — V 


i(caic)  x^,A,B,C 


(3t\ 

\BA> 


d2F 


) 


+ 


(  .  . 

Xl'yl(calc)»B»C  dyi(caic)  *j»A»B>C 


i(calc)  x^ ,A,B ,C 


(58) 


Similarly, 


B  x . ,y . ,  n  ,A,C 

i  i(o) 


B) 


BB  x . ,y . ,  1  v  ,A,C 

l  i(calc) 


x.  ,y 


,G 


B2f 


i i(calc) ’ 


A  ,C  '^^^-^i(calc) '  x  .  ,A,C 


-) 


( 


3F 


By 


-) 


i(calc)  x^,A,B,C 


( 


BF 


By 


i(calc)  x^,A,B,C 


b2f 


) 


+ 


(- .  , 

Vyl(calcrA’C  Syi(calc)  VA»B»C 

(  »  .f 

^i(calc)  x.  ,A,B,C 


(59) 


and 


J 

] 


J 

5 


' 
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i’yl(o) ’A>B 


(5) 


‘dC  x . ,y . .  .  v  ,A,B 

i(calc)  ? 


.(l 


S2f 


'oF\ 

AC)  .  AdCSy.,  . 

Vyl(calc)-A’B  l(caic)  VA>B 


-) 


( 


SF 


-) 


^yi(calc)  x^,A,B,C 

2 


i(calc)  x^,A,B,C 


.(: 


B2F 


Xi,yl(calc) ,A,B  5yj(calc)  Xj?A?B?C 

(^~)3 


i(calc)  x^,A,B ,C 


(60) 


Differentiating  equation  (49)  with  regard  to  B,  keeping  ,A,C 

fixed,  we  have 


xi’yi(0) ’B>C  xi’yi(o) ,A’C 


3_/3F' 

,oB\oA; 


( 


Xi’yi(calc>’B’C  xj’A’C 

SF  \ 

^yi(calc)  x . ,A,B ,C 


dF 


) 


Xi,yi(calc)  ,B,C  ^  aYi(ca;Lc)  x.  ,A,B,CJx,,A,C 


( 


dF 


Sy. 


-y 


i(calc)  x^,A,B,C 


(61) 


/dF\ 

Let  us  differentiate  (  r~J  with  regard  to  B,  keeping  x., 

\OA/  n  i 

Xi’yi(calc)-B>C 

A,  and  C  constant.  When  we  do  this,  we  get 


i ,yi(ealc) ,B,C  xi’A,C 


LdBVdA ) 


x 


i’yi(Calc) ,B,C  Xi,yi(calc) ,A’C 


+ 


/dyi(calc)\ 

LByi(calc)^Ay/xi,yi(.calc^  ,B,CJxi,A,B,C  x^AjC 


(62) 


■ 


' 


■ 


4i 
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j 


But 


] 

] 

] 

] 

] 


I 

- 


fdyi(calc)\ 

\  BB  / 


\BB  / 


x . ,y . /  N ,A,C 
1  i(calc)  ? 


x .  ,A  ,C 

l 


i(calc)  x^,A,B,C 


(63) 


and  if  we  substitute  equation  (63)  in  equation  (62)  ,  we  see  that 


B_/'M'\  1 

.3B\BA/  J 

x . ,y . f  N ,B  ,C  x . ,A,C 
l  y i(caic)  ’  5  i  ’  ’ 


(— ) 

\3A3B/ 


x 


i ’^i(caic) 


{ 


d2F 


BABy , 


-) 


(calc)  x  , 


(64) 


Now  if  we  differentiate 


( 


3F 


By . 


-) 


with  regard  to  B,  holding 


i(calc)  x , ,A,B ,C 


x_^,  A,  and  C  constant,  we  get 


n 

J 

fB  f  .  3F 


tefr-) 


_3B  \3y 


i(calc)  x.  ,A,B,C  x.,A,C 


] 

] 


[M 


3F 


By 


-) 


i(calc)  x.,A,B,C  x.,yi(calcrA,C 


r  a 

*14 


(sr^TT) 


i(calc)  •/i(calc)  x^,A,B,C  x^,A,B,C 


^^i(calc)\ 

BB  ) 


x ,  ,A,C 

l 


or,  substituting  equation  (63)  in  equation  (65), 


] 


(65) 


. 
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BF 


-) 


3B  Syi(calc)  x<ja,B,C  x.,A,C 


32F 


A 


^B^yi(calc)  x.,A,C 


d2F 


-) 


x.  ,y. 


i  i(caic) 


,A,C 


Syi(calc)  xi,A,B,C  (ay.,  .  N  ) 


^yi(calc)  x . ,A ,B ,C 


(66) 


Therefore,  if  we  substitute  equations  (64)  and  (66)  in  equation  (61) 
equation  (61)  is  expressible  as 


i!i) 

BABB  / 


Xi’yi(o) 


[  a2F 

\babb 


) 


x .  ,y 


i i(calc) 


>C 


'sf\  t  a2F 

Wxl’yj(caic)’A’cVaA3yi(calc)  ^..E.C 


( 


BF  \ 

^yi(calc)  x.,A,B,C 


^yi(calc)  x . ,A,B ,C 


k 


a2F 


BF 

^V^calc)  .B.CVaB9yi(calc)  4.  ,A,C 


-) 


i(calc)  x^,A,B,C 


(-) 

\3A/ 


<: 


a2F 


-) 


x .  ,y . ,  .  v,B,C  x.,y.,  -  x  ,A,C  By.,  ,  N  x.,A,B,C 

i  ^ i(calc)  ’ i  J i(calc)  y i(caic)  1 


( — — ) 
°yi(calc)  x.,A,B,C 


(67) 


. 


■■ .  a ' 


< 


i 


j  . 
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2 

,3  Y 


We  can  derive  expressions  for  (^3c)  and 


2 

>d  Y 


x . ,y .  ,  N  ,B 
l  i(o) 

\3B3C/  similar  to  equation  (67) .  The  results  are 

Vyi(o)  >A 


( 


2 

31, 


i'l 


SASC/ 


Vyi(o)’B 


(2lU \ 

\3A3C/ 

x 


( 


i ?yi(ealc) 


,G 


32F 


X 


3y 


-) 


r3T\ 

*i?yi(calc) ?A?B  yi(calc)  xi?B>C 


■) 


i(calc)  x4,A,B,C 


( 


3F 


■y 


^yi(calc)  x^AjB^C 


x 


( 


2 

a  f 


3C3y. 


•) 


l»yl(c.lc)’B’C  ^(calc)  x.,A,B 


i(calc)  x . ,A,B ,C 
i 


AX 


-) 


+ 


Vyi(calc)  >B’C^  Vyi(calc)  ,A?B  dyi(calc)  xj,A?B?C 


( 


3F  V 


^yi.(calc)  x^,AjB?G 


(68) 


2 

MW 


Vyi(e) ’A 


2 

a  f 


lx  /3I\  (. 

\aB3c J  .  \ac/  .  Assay.  (  ,  J  A  „ 

Vyl(ealc)/A  Vyl(ealc) >A?B  i(calc>  X<»A,C 


(■ 


X 


■). 


■) 


^yi(ealc)  X£,A,B,C 


( 


1L 


k 


AX 


■). 


) 


X.  ,A,B  ,G 


*i,yi(ealc) A»CNBCByi(eale)/x1,A,B 

: .  2  . 


^yi(eaie)  x.  ,A,B,C 


ac 


(: 


■) 


Xi’yi(eale)‘A’C  xl’yi(ealc)  ,A?B  ayl(gale)  VA-B’C 


( 


■) 


^yi(calc) 


(69) 


'>'■  5  '•  ,  :  ,  t>  * 


' 
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The  method  of  obtaining  a  least  squares  solution  for  the  con¬ 
stants  A,  B,  and  C  is  as  follows. 

We  have  n  pairs  of  the  observed  quantities,  x^  and  . 

1.  We  assume  values  for  A,  B,  and  C. 

2.  We  then  calculate  the  n  values  of  F^,  using  equation  (2). 

3.  We  next  calculate  the  n  values  of  y^caj. c^using  an  itera¬ 

tive  method  involving  equations  (42)  ,  (43)  ,  and  (44) . 

4.  We  next  calculate  the  n  values  of  from  equation  (4). 

2 

5.  We  next  calculate  the  n  values  of  Y  . 

6.  Using  designated  values  for  w  ,  we  next  calculate  R, 

yi(o) 

the  sum  of  the  weighted  squares  of  the  residuals,  using  equation  (5). 

7.  We  next  calculate  the  n  values  of  (dF/dy.,  ,  v)  by 

l lea lc7 

differentiation  of  the  analytical  expression  for  F,  keeping  x^,  A, 

B,  and  C  constant. 

8.  We  next  calculate  the  n  values  of  (dF/dA) ,  evaluated  at 

y.,  ,  v  ,  by  differentiation  of  the  analytical  expression  for  F, 

i(calc) 

keeping  x.,  y,,  .  B,  and  C  constant. 

r  i  J i(calc) 

9.  We  next  calculate  the  n  values  of  (dF/dB) ,  evaluated  at 

yi(calc) *  ^  differentiating  the  analytical  expression  for  F, 
keeping  x^  ,  A,  and  C  constant. 

10.  We  next  calculate  the  n  values  of  (dF/dC)  ,  evaluated  at 

y.(  .  ,  by  differentiation  of  the  analytical  expression  for  F, 

1 ^C3 1C  J 

keeping  x^,  y^(ca^c)  >  A,  and  ®  constant. 

2  2 

11.  We  next  calculate  the  n  values  of  (d  F/dA  ),  evaluated  at 

y.(  .  . ,  by  differentiating  the  analytical  expression  for  F  twice. 

1  \ C 3  J.C  / 


. . 

, 


£ 
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2  2 

12.  We  next  calculate  the  n  values  of  (3  F/3B  ),  evaluated  at 

y^(caic) >  by  differentiation  of  the  analytical  expression  for  F 
twice . 

2  2 

13.  We  next  calculate  the  n  values  of  (3  F/3C  ),  evaluated  at 

^i(calc) ’  ^y  differentiating  the  analytical  expression  for  F  twice. 

2  2 

14.  We  next  calculate  the  n  values  of  (3  F/By./  .  N )  by 

yi(calc)  J 

differentiating  the  analytical  expression  for  F  twice. 

2 

15.  We  next  calculate  the  n  values  of  (3  F/3A3B) ,  evaluated 

at  >  which  is  obtained  from  the  analytical  expression  for  F. 

2 

16.  We  next  calculate  the  n  values  of  (3  F/BABC) ,  evaluated 

at  y.j_(ca^c)  >  which  is  obtained  from  the  analytical  expression  for  F. 

2 

17.  We  next  calculate  the  n  values  of  (B  F/3B3C)  ,  evaluated  at 

^i(calc)  ’  *-s  obtained  from  the  analytical  expression  for  F. 

2 

18.  We  next  calculate  the  n  values  of  (3  F/By./  ,.370  which  is 

1  ^Ccl  1C  J 

obtained  from  the  analytical  expression  for  F. 

2 

19.  We  next  calculate  the  n  values  of  (B  F/By./  -  .  3B)  which 

l^CS 1C / 

is  obtained  from  the  analytical  expression  for  F. 

2 

20.  We  next  calculate  the  n  values  of  (3  F/By.,  -  .3C)  which 

i(calc) 

is  obtained  from  the  analytical  expression  for  F. 


21.  We  next  calculate  the  n  values  of  (3Y^/3A)  using  equation 


(49) 


22.  We  next  calculate  the  n  values  of  (dY  /SB)  using  equation 


(50)  . 


23.  We  next  calculate  the  n  values  of  (SY /SC)  using  equation 


(51)  . 


' 


' 
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24. 

(58) . 

25. 

(59)  . 

26. 

(60) . 

27. 
(67). 

28. 


2  2 

We  next  calculate  the  n  values  of  (d  Y^/BA  )  using  equation 


2  2 

We  next  calculate  the  n  values  of  (d  Y^/dB  )  using  equation 


2  2 

We  next  calculate  the  n  values  of  (B  Y_^/3C  )  using  equation 


We  next  calculate  the  n  values  of  (d  Y^/BAdB)  using  equation 


We  next  calculate  the  n  values  of  (3  Y^/3AdC)  using  equation 


(68). 

2 

29.  We  next  calculate  the  n  values  of  (d  Y^/3BdC)  using  equation 
(69). 


30. 

We 

next 

calculate 

al 

from  equation 

(19). 

31. 

We 

next 

calculate 

a2 

=  from  equation  (20)  . 

32. 

We 

next 

calculate 

a3 

=  c^  from  equation  (21) . 

33. 

We 

next 

calculate 

b2 

from  equation 

(22). 

34. 

We 

next 

calculate 

b3 

=  c 2  from  equation  (23) . 

35. 

We 

next 

calculate 

°3 

from  equation 

(24). 

36. 

We 

next 

calculate 

ml 

from  equation 

(25). 

37. 

We 

next 

calculate 

m2 

from  equation 

(26)  . 

38. 

We 

next 

calculate 

m3 

from  equation 

(27). 

39. 

We 

next 

calculate 

from  equation 

(31). 

40. 

We 

next 

calculate 

D2 

from  equation 

(32). 

41. 

We 

next 

calculate 

D3 

from  equation 

(33). 

42. 

We 

next 

calculate 

D,, 

from  equation 

(34). 
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43.  We  next  calculate  from  equation  (35), 

44.  We  next  calculate  D,  from  equation  (36). 

b 

45.  We  next  calculate  D from  equation  (37). 

46.  We  next  calculate  Dg  from  equation  (38). 

47.  We  next  calculate  from  equation  (39). 

48.  We  next  calculate  Dq  from  equation  (40). 

49.  We  next  calculate  AA  from  equation  (28). 

50.  We  next  calculate  AB  from  equation  (29). 

51.  We  next  calculate  AC  from  equation  (30). 

52.  We  now  return  to  step  1  and  calculate  kQ  +  AA,  where  Aq  is 

the  value  originally  assumed  in  step  1. 

53.  We  next  calculate  +  AB ,  where  is  the  value  originally 

assumed  in  step  1. 

54.  We  next  calculate  +  AC,  where  C^  is  the  value  originally 
assumed  in  step  1. 

55.  Using  (A  +  AA) ,  (Bq  +  AB) ,  and  (CQ  +  AC)  as  new  values  of 
A,  B,  and  C,  we  proceed  to  step  2  and  repeat  steps  2  through  6. 

56.  At  this  point,  we  compare  the  value  of  R,  the  sum  of  the 
weighted  squares  of  the  residuals,  with  the  initially  calculated 

value  of  R.  If  it  is  smaller,  we  proceed  to  step  1  and  repeat 

steps  7  through  55.  We  continue  to  repeat  the  iteration  until 

m  =  m?  =  nu  =  0  within  some  predetermined  small  quantity.  Our 
12  3 

final  values  of  A,  B,  and  C  are  our  least  squares  solution  for  these 


quantities . 


' 

'■ 
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If  in  step  56  the  sum  of  the  weighted  squares  of 
R,  is  larger  than  the  initial  value  of  R,  the  problem 
and  at  this  point,  a  technique  must  be  used  that  will 
vergence .  One  method  that  will  lead  to  a  smaller  sum 
negative  gradient  or  the  method  of  steepest  descent, 
of  a  vector  in  the  direction  of  the  negative  gradient 


the  residuals, 

is  diverging, 
lead  to  con- 
of  R  is  the 

The  components 
are  given  by 


o 


Thus ,  if  we  take 


where  k  is  a  positive  constant,  we  will  move  in  the  direction  of  the 
negative  gradient;  k  has  been  called  the  size  of  the  step.  We 
determine  the  size  of  the  step  in  the  following  way.  We  expand  R 
in  a  Taylor's  series  expansion,  retaining  terms  through  the  second 


derivatives.  Thus, 
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R  = 


(73) 


We  now  substitute  in  equation  (73)  for  AA ,  AB ,  and  AC  from  equations 
(70),  (71),  and  (72).  The  result  is 


(74) 


We  now  differentiate  equation  (74)  with  regard  to  k,  set  the 
derivative  equal  to  zero,  and  solve  for  k.  The  result  is 

k  = 


(75) 


From  equation  (5)  and  equations  (19)  -  (27)  ,  it  is  possible 


show  that 


®° 

=  -2m. 

(76) 

®° 

= 

(77) 

v  (®° 

=  -2m3 

(78) 

$>■ 

"  2al 

(79) 

“  2t2 

(80) 

<$> 

=  2c3 

(81) 

(  *2r)°  = 
\dA3B/ 

2a2 

2bl 

(82) 

/  a2R\° 

\dASC ) 

2a3 

2c . 
i 

(83) 

l  ^R)°  - 

VdBdC/ 

2b3  - 

2c2 

(84) 

Substituting  equations  (76)  -  (84)  into  equation  (75)  ,  we  have 
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We  see,  therefore,  that  if  the  Newton-Raphson  method  is  used  in 
setting  up  the  normal  equations,  the  size  of  the  step  to  be  taken 
in  the  direction  of  the  negative  gradient  can  be  evaluated  very 
simply  from  the  coefficients  appearing  in  the  normal  equations, 
provided  the  calculation  of  k  leads  to  a  positive  quantity.  If 
equation  (85)  leads  to  a  negative  quantity,  this  means  that  the 
curvature  of  the  surface  is  such  that  the  trial  solution  is  near 
a  maximum  and  not  a  minimum.  Under  these  conditions,  the  negative 
value  of  k  must  be  ignored  and  positive  values  of  k  explored  on 
a  trial  basis. 

Of  course,  the  same  formal  calculation  can  be  made  if  the 
Gauss-Newton  method  is  used  to  set  up  the  normal  equations.  How¬ 
ever,  if  one  is  in  a  region  of  divergence,  this  means  that  the 
trial  solution  is  far  from  the  true  answer.  Under  these  conditions, 
the  residuals  will  be  large  and  the  second  summation  in  the  a's, 
b's,  and  c's,  involving  the  second  derivative  terms,  will  be  of 
importance  compared  to  the  first  term  in  the  summation.  We  there¬ 
fore  believe  that  if  it.  is  necessary  to  use  the  negative  gradient 
method,  it  is  better  to  use  the  Newton-Raphson  method  in  setting 
up  the  normal  equations  and  in  calculating  the  size  of  the  step. 

If  our  problem  is  diverging  after  the  first  iteration,  we 
calculate  AA,  AB ,  and  AC  from 


AA  = 

2km^ 

(86) 

AB  = 

2km2 

(87) 

AC  = 

2km^ 

(88) 

f 


■ 
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with  k  calculated  from  equation  (85).  Using  Aq  +  AA,  +  AB , 
and  C^  +  AC  as  new  values  of  our  undetermined  constants,  A,  B, 
and  C,  we  then  return  to  step  2  of  the  iteration.  For  each  new 
value  of  A,  B,  and  C,  we  solve  the  normal  equations  and  calculate 
a  new  value  of  R.  If  we  are  diverging,  we  continue  with  the  nega¬ 
tive  gradient  method  until  the  region  of  convergence  is  reached. 

As  soon  as  this  happens,  we  drop  the  negative  gradient  method 
and  iterate  by  solving  the  normal  equations  for  AA,  AB ,  and  AC. 

A  scheme  such  as  this  should  always  lead  to  convergence. 

Although  the  above  scheme  should  always  lead  to  convergence, 
the  negative  gradient  method  may  be  tediously  slow  in  entering 
the  region  of  convergence  for  the  solution  of  the  normal  equations. 
Under  these  conditions,  other  schemes  can  be  tried  which  may  enter 
the  region  of  convergence  more  rapidly  than  the  negative  gradient 
method . 

Some  of  these  methods  are:  (1)  the  Hartley  (_3)  method;  (2)  a 
modification  of  the  Hartley  method  due  to  Strand,  Kohl,  and  Bonham 
(_8)  ;  and  (3)  the  method  of  Marquardt  (4) . 

We  do  not  have  enough  experience  to  judge  the  relative  merits 
of  these  various  methods.  In  our  applications  so  far,  we  have  not 
been  troubled  by  lack  of  convergence. 

CALCULATION  OF  VARIANCES  AND  COVARIANCES 

With  the  value  of  our  constants  determined,  the  remaining 
questions  to  be  answered  are:  (I)  What  are  the  variances  and 
covariances  of  the  constants  evaluated?  (2)  What  are  the  variances 


f 
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of  the  calculated  y  ' s  and  of  any  other  calculated  y  that  reduces 
F  to  zero? 

To  answer  these  questions,  we  apply  the  law  for  the  propaga¬ 
tion  of  errors.  This  law  states  that  if  we  have  a  quantity  or 
function,  Q,  that  is  a  function  of  the  independent  quantities,  y^, 
y^,  ^3’  *■*  5  t^ien  t^rie  variance  of  Q  is  given  by 


n 


M. 


•  By.  /  \ 

1=1  ^o) 


) 


y. 


(89) 


i(o) 


2  2 

where  is  the  variance  of  Q,  and  S 

Q  y. 


is  the  variance  of  y. 


i(o) 


i(o) 


The  value  of  the  constant.  A  that  we  have  evaluated  is  a 


function  of  all  of  the  observed  x^'s  and  of  all  of  the  observed 
y.'s.  Since  we  have  assumed  there  are  no  random  errors  in 

l 

the  x^'s,  the  variances  of  the  x/s  are  zero.  Then  the  expression 
for  the  variance  of  A  is  given  by  the  equation 


■  i  fe^): 

i=l  i(o) 


y 


i(°) 


(90) 


and  there  will  be  an  equation  similar  to  equation  (90)  for  eval¬ 
uating  the  variance  of  B  and.  of  C, 

To  evaluate  equation  (90),  we  must  evaluate  (8A/8y..  .  )  for 

i(o) 

each  Y-/  multiply  this  quantity  by  S  ,  square  the  product, 

Yi(o) 

and  then  sum  the  product,  over  all  of  the  observed  y^’s. 

Now  we  have  a  total  of  n  pairs  of  the  observed  quantities  x., 


yi-  Then  our  constants  to  be  evaluated  are  determined  by  the 


. 
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solutions  of  equations  (6) ,  (7) ,  and  (8) .  Now  suppose  we  change 

one  of  the  y\'s,  ^2(0)  sa^’  t0  -^2(o)  +  ^2'  ^en  on  solving  equa¬ 
tions  (6) ,  (7) ,  and  (8) ,  we  would  get  new  values  of  (A  +  AA) , 

(B  +  AB)  ,  and  (C  +  AC)  for  our  constants.  Then  we  can  calculate 


5A 


By 


2(o) 


AA 

Ay. 


SB 


Sy 


2(0) 


AB 

Ay. 


SC 


By 


2(o) 


AC 

Ay0 


(91) 


(92) 


(93) 


This  means  that  when  is  changed  by  a  small  amount,  equa¬ 

tions  (6) ,  (7) ,  and  (8)  must  still  hold  exactly.  Mathematically, 
this  means  that 


_iL_  /"M)  =  0 

3y . ,  .  V3A 1  „  „ 

l(o)  xi’yi(o)’B,C 


(94) 


© 


3yi(o)  •'”!ti-,yi(o),A1c 


=  0 


(95) 


By, 


i(°) 


© 


=  0 


Xi-yi(o) >A'B 


(96) 


All  of  the  Y.'s  are  explicit  functions  of  the  y.,  N's,  the  x.'s. 

1  r  7 1(0)  1 

and  the  constants  A,  B,  and  C.  In  the  application  of  equations  (94), 
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(95)  ,  and  (96)  ,  the  derivatives  (SR/dA) 


(9R/BB) 


Xi’yi(o)  ’B’C’ 


xi ,yi(o) 


,A,C’  anc*  (SR/ciC)^  ,y^(0)  >A,B  are  to  be  considered 


functions  of  all  of  the  y.,  N 1 s  and  of  the  constants  evaluated,  with 

i(o) 

A,  B,  and  C  being  functions  of  all  of  the  y^^'s.  Under  these 
conditions , 


- ( ~) 

dy  VSA/  -D  r> 

__i(o)  xi>yi(0)  ’B^. 


xi’yj^i 


/BR\  f  5A  \ 

3A  Xi’yi(o) ,B,C  l(o)  yj#i 


-) 

w 


SBSA/  ASy., 

xi’yi(o)’C  l(o) 


+ 


\dCcw 


xi>yi(o) 


+ 


x  . ,  y, ,  s  ,b  ,c 
X  r(o) 


3y 


i(o) 


.y^-  ,A,B,C 


=  0  (97) 


'SR' 


3yi(o)  V3B-x  v  AC 
_U°;  xi’yi(o)  >A± 


x . ,y .  , . 

i  jfi 


(JL) 


■SB  x.,yi{o);A,C  1(0)  yj)6. 


+ 


oA  \ 


(A 

VdAbB/  pVdy., 

xryKo)’c  l(o)  yj^i 


.G 


+  (-JLS) 

VbCBB / 


,SB 


x . ,y . ,  >.  ,A,C 
l^iCo) 


—  ^yi(o)  — 


xi>yj/i’A’2£ 


=  0  (98) 
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Equations  (97)  ,  (98)  ,  and  (99)  contain  terms  involving  the 

derivative  of  each  constant  with  respect  to  y^^  and  can  solved 

by  elementary  algebra  for  the  derivatives  of  the  constants  with 

respect  to  y^^  .  These  derivatives  are  then  to  be  multiplied  by 

S  ,  the  product  squared,  and  then  the  squared  product  is  to  be 

yi(o) 

summed  over  ail  of  the  observed  y.,  '  s .  These  sums  give  us  the 

i(o) 

variance  of  each  constant  evaluated. 

Differentiating  R,  given  by  equation  (5),  by  the  constants  A, 
B,  and  C  keeping  x^  and  y^^  constant,  we  have 


Vy, 


i(o) 


,BSC 


,B  ,C 


0  (99) 


(100) 
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Vyi(o)  ,A,C 


n 

2IW 

i=l 


yi(°) 


,dY.< 

a 


(101) 


Vyi(o)  >A’C 


xi’yi(o) -A>B 


n 


r  /bY-\ 

Zj  Wy  . ,  JAW) 


2  )  w  Y 

i=l  y^°>  1  '  Vyi(o)’A’B 


(102) 


Differentiating  equation  (100)  with  regard  to  each  constant,  we  have 


82R 


) 


n 


SA  Xi’yi(o)  ,B,C 


»  2  y  w 

1-1  yi(°) 


/SY ,v  2 

(sr) 


+  Y 


2 

,3  Y.\ 

AA 


xi,yi(o) ,B,C  3A  Vyi(o)’B’C 


(103) 


Xi’yi(o)  ,B>C 


Xi’yl(o)  >A’C 


A  /dY.v  /dY .  v 

2  Z,  wy.r  Abb”)  Aba") 

1=1  l(o)  Xl’yl(o)  ,A,C  Xi’yl(o)’B,C 


n 


r*i 

+  2  )  w  Y. 
i-l  Ao) 


dB 


,BY.N 

(s1) 


xi’yi(o)’B>£ 


x . ,y . ,  s ,A,C 
i i(o)  * 


(104) 


xi’yi(o)’B-i. 


Xi'yi(o)  >A’B 


n 

2  E  w 

1=1  yi(o)  xl’yl(o)  ,A,B  '  xi,yi(o)  ,B,C 


& 


(-1) 
\BA  / 


n 

+  2  w 


i~i  YAo)Yi 


BC 


■  BY 


iw) 


Xl’yl(0)’B£ 


xl’yi(o) -A’B 


(105) 
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Differentiating  equation  (100)  with  regard 
keeping  A,  B,  and  C  constant,  we  have 


to  a  s  ing  ie  y .  ,  .  , 
1(0) 


(106) 


The  right-hand  side  of  equation  (106)  reduces  to  a  single  term,  since 
a  single  only  appears  in  one  term  in  the  summation. 

Now  from  equation  (4) 


Y. 

l 


^i(o)  ^i(calc) 


(4) 


so  that 


( 


3Y 


3  v  ^ 

yi(o)  xi,y_.^i,A,B,C 


=  1 


(107) 


and  it  follows  that 


a  /  3Y.  * 

A.  (—JlJ) 

3A  \3y . ,  N „ 
^(o)  ^i,y^i,APBaC 


=  0 


x . ,y .  ,  x  ,B  SC 
i  i(o)  ’  s 


(108) 


Substituting  equations  (107)  and  (108)  into  equation  (106)  ,  we  have 


40 


<dAj 


x . ,y  .  /  N  ,B ,C 


I—  3yi(o) 


=  2w 


Wi’A’B’C 


,BY.* 
yi(o)  3A 


(109) 


Substituting  equations  (103),  (104),  (105),  and  (109)  in  equation 
(97) ,  we  get 
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(r^H  7* 

yK°)  yJ^1  i-i  yi(°) 


/Hi'2 


32Y 


\5A  )*  +  Yl(~21) 

xi’yi(o)’B»C  SA  xl’yl(o)’B,C 


( 


as\ 


ayi(o)  y 


n 


BY 


yw  (!!i^  p.) 

L.  y ABB  /  ,  A  BA  /  „  r 

1=1  i(o)  '  xl’yl(o) ,A,C  xi’yl(o)’B’C 


n 


ra  fhY i\ 

Yj  /  iLbb  \BA  /  «  v  AO 

i-1  i(o'  Vyi(o)  ,BsC  xi,yi(o) ,AsC 


I 


+  >  w 


J 


( . W . ) 

'3yi(e/y 


J  H 


n 


BY 


,BY.n 


ly,  AiA  v  .  .(sA 

1-1  1(o)  xl’yi(o)  ,A’B  xl’yi(o) 


n 


+  T  w  Y 

i-i  y*<e>  1 


i. 

,BC  VBA  / 


K4  A 


x4  A#® 


+  w 

yl(e) '®A  y.y^.B.C 


»  o 


(110) 
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Upon  differentiating  equations  (101)  and  (102)  with  regard  to 
a  single  y^^ >  we  obtain  two  expressions  similar  to  equation  (110) 
We  will  write  equation  (110)  and  the  two  other  equations  as 


l\c 


BA  \ 


a,l,By .  ,  J 


+  b 


(-J*-) 

l\By . /  J 


,  ,  +  c  )  =  n  (111) 

^(o)  yJjti  U(o)  yjH 


+b,(-as_)  •  +  c,(r-§S-)  -  n,  (112) 

2'3yi(o)  y<4j  A3yi(o)4.^  Aayi(o/y^. 


BA 


“sW^Tt)  +  b3(ay~r)  +  c3(s^7t)  =  n3  (u3) 

yi(o)  yJ)H  1(0)  yj!tl  ^(°)  yj/i 


where 


n 


a.  -  Vw 

1  4  yi(°) 


BY 2 


(sa  ) 


+  Y 


A) 


xi’yi(o)  ,B’C  *  'SA  xi’yl(o)’B>C 


(114) 


n 


w 


i=l 


yi(°) 


© 


,3Y.\ 

k1) 


Vyi(o)  ’A,c  Vyi(o)  ’B,c 


n 

+  I w 

i=l 


2 

BY., 


A  ”y. ,  , Yi(&B9a) 


yi(o) 


xi’yi(o) ,C 


(115) 


a3  C1 


n 


i=l 


A) 

\BA  /  r,  r. 

Xl’yi(o)  ,A,B  xi’yl(o)>B’C 


v-  ,32Y.N 

+  IWy.,  Yi(i-^ 


n 


L  y.f  v  i\BCBA/ 

1=1  j-w  vyi 


1(0) 


,B 


(116) 
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(117) 


,A,B 


i(°) 


(119) 


(120) 


(121) 


(122) 


Notice  that  the  a's,  b's,  and  c's  defined  by  equations  (114)  - 


(119)  are  of  the  same  form  as  the  a's,  b's,  and  c's  defined  by 


. 
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equations  (19)  -  (24)  which  appear  in  our  normal  equations.  The 
difference  between  these  two  definitions  is  that  equations  (114)  - 
(119)  apply  when  the  least  squares  solution  is  obtained  while 


equations  (19)  -  (24)  apply  to 

c's  given  by  equations  (114)  - 

values  of  the  coefficients  in 

iteration. 

Solving  equations  (111) , 

(dA/dy^^)  ,  (dB/dy^^)  ,  and 

trial  solutions.  The  a's,  b's,  and 

(119)  can  be  considered  as  the 

the  normal  equations  for  the  final 

(112) ,  and  (113)  simultaneously  for 

(dC/dy^^)  ,  we  get 

d(  bA  )  ■ 

. o\oy . t  N / 
i(o) 

Vi 

+  ^>2n2  +  D3n3 

(123) 

D  (-  aB  )  = 
oVayi(o)' 

D4nl 

+  D5n2  +  D6n3 

(124) 

D  L  3C  )  - 

Aoyi(o/ 

Vl 

+  Dgn2  +  D^n^ 

(125) 

where 

h  ’ 

b2C3  ■ 

b3°2 

(126) 

D2  = 

Vl  ■ 

'  blC3 

(127) 

D3  " 

blC2  ' 

-  b2cx 

(128) 

D4  “ 

a3C2 

a2C3 

(129) 

D5  " 

alC3 

a3Cl 

(130) 
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D6 

a2Cl  "  alC2 

(131) 

D7 

~  a2^3  ~  a3°2 

(132) 

D8 

=  a3bl  "  a 1 b 3 

(133) 

D9 

alb2  '  a2bl 

(134) 

D 

o 

alDl  +  a2D2  +  a3D3 

(135) 

Notice  that  the  D's  defined  by  equations  (126)  -  (135)  are  of 
the  same  form  as  the  D's  defined  by  equations  (31)  -  (40)  and,  for 
the  last  iteration,  when  the  least  squares  solution  is  obtained,  they 
will  be  identical. 

2 

Squaring  equation  (123)  and  multiplying  by  S  ,  we  have 

yi(o) 


2  2C2  ,  n2  2S2  ,  n2  2C2 

D.n-S  +  D.n_S  +  D^n^S 

1  1  yl(o)  2  2  yi(o)  3  3  yi(o) 
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1212  y1(o)  1313  y1(0) 


2 
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2  3  2  3  y 


l(o) 


(136) 


But  from  equation  (120) 


(137) 
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Now 


y 


.  ,  .  w 

1(0)  y. 


(138) 


i(o) 


where  L  is  a  constant.  Substituting  equation  (138)  in  equation  (137), 
we  find  that 
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Similarly,  we  find  that 
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(144) 


Substituting  equations  (139), 

(144)ilnto  “•>“«<>»  <136)  and  then 
yi(o)  s’  We  have 


(140)’  (WI),  (142),  a43),  and 

summing  over  all  of  the  observed 
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Squaring  equation  (124),  multiplying  by  S  and  then  summing, 

yi(o) 


we  find  that 
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Similarly,  from  equation  (125)  ,  we  find  that 
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For  the  covariances,  we  find  from  equations  (123),  (124),  and 
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and  finally, 


■^Xi^y2 


3yi(o)A3yi(o/  yi(o) 


s2  =  i- 

BG  d2 

O 


n 


V?  L  Wy 


/SY.X2 

(sr) 


i=l  *-  (°)  xi  -,yi  (o)  ,B 


n 


BY.V2 


+  V8IWy..  .(sT) 

1=1  l(0)  Vyi(o)’A>C 


n 


+ 


/BY.X2 

,(sr) 


V9 1" 


1=1  ’  1(°)  Xi’yi(o)’A’B 


+(D5D9+  D6D8} 


n 

X  Wy.  .  < 
1=1  1(°> 

\<5k  ) 

n 

X  wy.  ,  < 
1=1  ^ 

,ar) 

n 

,3Y  S 

Xv,  ( 

i=l  i(o) 

5:;) 

(S 

Xi’yi(o)  ’B’C  xi>Yi(o)’A>C 


SY.v 


hr) 


Xi,yi(o),B,C  ~  Xi’yi(o)>A’B 


,BY. 

.(sc' 


) 


‘  i 5  y  i  (  o) 


x.,y.  ,  .,ASB 
i7i(o )’  5 


(150) 


- 

» 


52 


Equations  (145),  (146),  (147),  (148),  (149),  and  (150)  allow 
us  to  calculate  the  variances  and  covariances  of  the  constants  eval¬ 
uated  . 

Now  Dq  is  the  value  of  the  determinate  of  the  coefficients  that 
appear  in  the  final  normal  equations.  In  a  linear  problem,  it  is 
found  that  may  be  factored  from  the  right-hand  side  of  equations 
(.145)  -  (150)  so  that  the  expressions  for  the  variances  and  covar¬ 
iances  only  involve  Dq  to  the  first,  power  in.  the  denominator.  This 
is  not  true  for  a  non-linear  problem  as  can  be  shown  from  the  exact 
derivation  given  above.  The  present  method  in  use  in  calculating 
variances  and  covariances  is  therefore  only  an  approximation.  How 
good  this  approximation  is  can  only  be  decided  by  testing  it  against 
the  mathematically  exact  calculation  given  above.  Since  this  is 
true,  we  prefer  to  calculate  variances  and  covariances  using  the 
mathematically  exact  equations  given  above. 

We  now  answer  the  question,  how  do  we  calculate  the  variance, 
of  a  calculated  y  which  reduces  the  function  F  to  zero  for  a  given 
x?  The  answer  to  this  question  is  obtained  as  follows:  y  is  a 
function  of  x,  and  through  the  constants  evaluated,  is  a  function 
of  all  of  the  observed  y^.^'s  and  x.'s.  When  we  apply  the  law  for 
the.  propagation  of  errors,  we  have 
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We  calculate 


& 


from  equation  (1) .  Differentiating 


equation  (1)  ,  we  have 
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Solving  equation  (152)  for  ('^y^  ) 
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Squaring  equation  (153),  multiplying  by  S  ,  and  then  summing 

Yi(o) 

over  all  the  observed  y  's,  we  obtain 
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where  the  variances  and  covariances  in  equation  (154)  are  to  be  calcu¬ 
lated  from  equations  (145),  (146),  (147),  (148),  (149),  and  (150). 
Equation  (154)  is  the  general  formula  for  calculating  the  variance 
of  a  calculated  y,  regardless  of  the  functional  relationship  between 
y  and  x  and  the  parameters,  A,  Bs  and  C. 

This  concludes  the  derivations  of  the  formulas  used  by  us  in 
the  solution  of  general  non-linear  least  squares  problems. 

In  a  later  paper,  we  will  apply  these  general  formulas  to  the 
reduction  of  PVT  data  obtained  by  the  Burnett  method. 
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